Skip to content

[SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example#25956

Closed
MaxGekk wants to merge 1 commit intoapache:masterfrom
MaxGekk:fix-expr-examples-checks
Closed

[SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example#25956
MaxGekk wants to merge 1 commit intoapache:masterfrom
MaxGekk:fix-expr-examples-checks

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Sep 28, 2019 •
edited
Loading

What changes were proposed in this pull request?

In the PR, I propose to clone Spark session per-each expression example. Examples can modify SQL settings, and can influence on each other if they run in the same Spark session in parallel.

Why are the changes needed?

This should fix test failures like this checking of the Like example:

org.apache.spark.sql.AnalysisException: the pattern '\%SystemDrive\%\Users%' is invalid, the escape character is not allowed to precede 'U';
      at org.apache.spark.sql.catalyst.util.StringUtils$.fail$1(StringUtils.scala:48)
      at org.apache.spark.sql.catalyst.util.StringUtils$.escapeLikeRegex(StringUtils.scala:57)
      at org.apache.spark.sql.catalyst.expressions.Like.escape(regexpExpressions.scala:108)

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running check outputs of expression examples in org.apache.spark.sql.SQLQuerySuite

@MaxGekk MaxGekk mentioned this pull request Sep 28, 2019
Closed
@MaxGekk MaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP] Clone SparkSession per each function example [WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example Sep 28, 2019
@HeartSaVioR HeartSaVioR mentioned this pull request Sep 28, 2019
Closed
@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111521 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111522 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111523 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@MaxGekk MaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example [SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example Sep 28, 2019
@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111526 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111528 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

@HyukjinKwon @wangyum @dongjoon-hyun Take a look at this PR, please.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111530 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111533 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

HyukjinKwon commented Sep 28, 2019

Merged to master.

@HeartSaVioR HeartSaVioR mentioned this pull request Sep 28, 2019
Closed
@MaxGekk MaxGekk deleted the fix-expr-examples-checks branch October 5, 2019 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@srowen srowen srowen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MaxGekk @SparkQA @HyukjinKwon @srowen